Understanding the stochastic dynamics of sequential decision-making processes: A path-integral analysis of multi-armed bandits.

The multi-armed bandit (MAB) model is one of the most classical models to study decision-making in an uncertain environment. In this model, a player chooses one of K possible arms of a bandit machine to play at each time step, where the corresponding arm returns a random reward to the player, potentially from a specific unknown distribution. The target of the player is to collect as many rewards as possible during the process. Despite its simplicity, the MAB model offers an excellent playground for studying the trade-off between exploration vs exploitation and designing effective algorithms for sequential decision-making under uncertainty. Although many asymptotically optimal algorithms have been established, the finite-time behaviors of the stochastic dynamics of the MAB model appear much more challenging to analyze due to the intertwine between the decision-making and the rewards being collected. In this paper, we employ techniques in statistical physics to analyze the MAB model, which facilitates the characterization of the distribution of cumulative regrets at a finite short time, the central quantity of interest in an MAB algorithm, as well as the intricate dynamical behaviors of the model. Our analytical results, in good agreement with simulations, point to the emergence of an interesting multimodal regret distribution, with large regrets resulting from excess exploitation of sub-optimal arms due to an initial unlucky output from the optimal one.

A high-frequency mobility big-data reveals how COVID-19 spread across professions, locations and age groups.

As infected and vaccinated population increases, some countries decided not to impose non-pharmaceutical intervention measures anymore and to coexist with COVID-19. However, we do not have a comprehensive understanding of its consequence, especially for China where most population has not been infected and most Omicron transmissions are silent. This paper aims to reveal the complete silent transmission dynamics of COVID-19 by agent-based simulations overlaying a big data of more than 0.7 million real individual mobility tracks without any intervention measures throughout a week in a Chinese city, with an extent of completeness and realism not attained in existing studies. Together with the empirically inferred transmission rate of COVID-19, we find surprisingly that with only 70 citizens to be infected initially, 0.33 million becomes infected silently at last. We also reveal a characteristic daily periodic pattern of the transmission dynamics, with peaks in mornings and afternoons. In addition, by inferring individual professions, visited locations and age group, we found that retailing, catering and hotel staff are more likely to get infected than other professions, and elderly and retirees are more likely to get infected at home than outside home.

Bilevel optimization in flow networks: A message-passing approach.

Optimizing embedded systems, where the optimization of one depends on the state of another, is a formidable computational and algorithmic challenge, that is ubiquitous in real world systems. We study flow networks, where bilevel optimization is relevant to traffic planning, network control, and design, and where flows are governed by an optimization requirement subject to the network parameters. We employ message passing algorithms in flow networks with sparsely coupled structures to adapt network parameters that govern the network flows, in order to optimize a global objective. We demonstrate the effectiveness and efficiency of the approach on randomly generated graphs.

Scalable node-disjoint and edge-disjoint multiwavelength routing.

Probabilistic message-passing algorithms are developed for routing transmissions in multiwavelength optical communication networks, under node- and edge-disjoint routing constraints and for various objective functions. Global routing optimization is a hard computational task on its own but is made much more difficult under the node- and edge-disjoint constraints and in the presence of multiple wavelengths, a problem which dominates routing efficiency in real optical communication networks that carry most of the world's internet traffic. The scalable principled method we have developed is exact on trees but provides good approximate solutions on locally treelike graphs. It accommodates a variety of objective functions that correspond to low latency, load balancing, and consolidation of routes and can be easily extended to include heterogeneous signal-to-noise values on edges and a restriction on the available wavelengths per edge. It can be used for routing and managing transmissions on existing topologies as well as for designing and modifying optical communication networks. Additionally, it provides the tool for settling an open and much-debated question on the merit of wavelength-switching nodes and the added capabilities they provide. The methods have been tested on generated networks such as random-regular, Erdos Rényi, and power-law graphs, as well as on optical communication networks in the United Kingdom and United States. They show excellent performance with respect to existing methodology on small networks and have been scaled up to network sizes that are beyond the reach of most existing algorithms.

Optimally coordinated traffic diversion by statistical physics.

Road accidents or maintenance often lead to the blockage of roads, causing severe traffic congestion. Diverted routes after road blockage are often decided individually and have no coordination. Here, we employ the cavity approach in statistical physics to obtain both analytical results and optimization algorithms to optimally divert and coordinate individual vehicle routes after road blockage. Depending on the number and the location of the blocked roads, we found that there can be a significant change in traveling path of individual vehicles, and a large increase in the average traveling distance and cost. Interestingly, traveling distance decreases but traveling cost increases for some instances of diverted traffic. By comparing networks with different topology and connectivity, we observe that the number of alternative routes plays a crucial role in suppressing the increase in traveling cost after road blockage. We tested our algorithm using the England highway network and found that coordinated diversion can suppress the increase in traveling cost by as much as 66% in the scenarios studied. These results reveal the advantages brought by the optimally coordinated traffic diversion after road blockage.

Futility of being selfish in optimized traffic.

Optimizing traffic flow is essential for easing congestion. However, even when globally optimal, coordinated, and individualized routes are provided, users may choose alternative routes which offer lower individual costs. By analyzing the impact of selfish route choices on performance using the cavity method, we find that a small ratio of selfish route choices improves the global performance of uncoordinated transportation networks but degrades the efficiency of optimized systems. Remarkably, compliant users always gain in the former and selfish users may gain in the latter, under some parameter conditions. The theoretical results are in good agreement with large-scale simulations. Iterative route switching by a small fraction of selfish users leads to Nash equilibria close to the globally optimal routing solution. Our theoretical framework also generalizes the use of the cavity method, originally developed for the study of equilibrium states, to analyze iterative game-theoretical problems. These results shed light on the feasibility of easing congestion by route coordination when not all vehicles follow the coordinated routes.

Global benefit of randomness in individual routing on transportation networks.

By introducing a simple model based on two-dimensional cellular automata, we reveal the relationship between the routing strategies of individual vehicles and the global behavior of transportation networks. Specifically, we characterize the routing strategies by a single parameter called path-greediness, which corresponds to the tendency for individuals to travel via a shortest path to the destination. Remarkably, we found that the effective dimension of the system is reduced when the congested states emerge. We also found that a high individual tendency to travel via the shortest path does not necessarily shorten the average journey time, as the system may benefit from less greedy routing strategies in congested situations. Finally, we show that adaptive routing strategies outperform controlled strategies in the free-flow state but not in the congested state, implying that controlled strategies may increase coordination among vehicles and are beneficial for suppressing traffic congestion.

Coordinating dynamical routes with statistical physics on space-time networks.

Coordination of dynamical routes can alleviate traffic congestion and is essential for the coming era of autonomous self-driving cars. However, dynamical route coordination is difficult and many existing routing protocols are either static or without intervehicle coordination. In this paper, we first apply the cavity approach in statistical physics to derive the theoretical behavior and an optimization algorithm for dynamical route coordination, but they become computationally intractable as the number of time segments increases. We therefore map static spatial networks to space-time networks to derive a computational feasible message-passing algorithm compatible with arbitrary system parameters; it agrees well with the analytical and algorithmic results of the conventional cavity approach and outperforms multistart greedy search in saving total travel time by as much as 15% in simulations. The study sheds light on the design of dynamical route coordination protocols and the solution to other dynamical problems via static analytical approaches on space-time networks.

Entropy Inflection and Invisible Low-Energy States: Defensive Alliance Example.

Lower temperature leads to a higher probability of visiting low-energy states. This intuitive belief underlies most physics-inspired strategies for addressing hard optimization problems. For instance, the popular simulated annealing (SA) dynamics is expected to approach a ground state if the temperature is lowered appropriately. Here, we demonstrate that this belief is not always justified. Specifically, we employ the cavity method to analyze the minimum strong defensive alliance problem and discover a bifurcation in the solution space, induced by an inflection point in the entropy-energy profile. While easily accessible configurations are associated with the lower-free-energy branch, the low-energy configurations are associated with the higher-free-energy branch within the same temperature range. There is a discontinuous phase transition between the high-energy configurations and the ground states, which generally cannot be followed by SA. We introduce an energy-clamping strategy to obtain superior solutions by following the higher-free-energy branch, overcoming the limitations of SA.

Slow spin dynamics and self-sustained clusters in sparsely connected systems.

To identify emerging microscopic structures in low-temperature spin glasses, we study self-sustained clusters (SSC) in spin models defined on sparse random graphs. A message-passing algorithm is developed to determine the probability of individual spins to belong to SSC. We then compare the predicted SSC associations with the dynamical properties of spins obtained from numerical simulations and show that SSC association identifies individual slow-evolving spins. Studies of Erdos-Renyi (ER) and random regular (RR) graphs show that spins belonging to SSC are more stable with respect to spin-flip fluctuations, as suggested by the analysis of fully connected models. Further analyses show that SSC association outperforms local fields in predicting the spin dynamics, specifically the group of slow- and fast-evolving spins in RR graphs, for a wide temperature range close to the spin-glass transition. This insight gives rise to a powerful approach for predicting individual spin dynamics from a single snapshot of an equilibrium spin configuration, namely from limited static information. It also implies that single-sample SSC association carries more information than local fields in describing the state of individual spins, when little information can be extracted from the system's topology.

Evolving power grids with self-organized intermittent strain releases: An analogy with sandpile models and earthquakes.

The stability of powergrid is crucial since its disruption affects systems ranging from street lightings to hospital life-support systems. While short-term dynamics of single-event cascading failures have been extensively studied, less is understood on the long-term evolution and self-organization of powergrids. In this paper, we introduce a simple model of evolving powergrid and establish its connection with the sandpile model and earthquakes, i.e., self-organized systems with intermittent strain releases. Various aspects during its self-organization are examined, including blackout magnitudes, their interevent waiting time, the predictability of large blackouts, as well as the spatiotemporal rescaling of blackout data. We examined the self-organized strain releases on simulated networks as well as the IEEE 118-bus system, and we show that both simulated and empirical blackout waiting times can be rescaled in space and time similarly to those observed between earthquakes. Finally, we suggested proactive maintenance strategies to drive the powergrids away from self-organization to suppress large blackouts.

Predicting the future trend of popularity by network diffusion.

Conventional approaches to predict the future popularity of products are mainly based on extrapolation of their current popularity, which overlooks the hidden microscopic information under the macroscopic trend. Here, we study diffusion processes on consumer-product and citation networks to exploit the hidden microscopic information and connect consumers to their potential purchase, publications to their potential citers to obtain a prediction for future item popularity. By using the data obtained from the largest online retailers including Netflix and Amazon as well as the American Physical Society citation networks, we found that our method outperforms the accurate short-term extrapolation and identifies the potentially popular items long before they become prominent.

Recovery of infrastructure networks after localised attacks.

The stability of infrastructure network is always a critical issue studied by researchers in different fields. A lot of works have been devoted to reveal the robustness of the infrastructure networks against random and malicious attacks. However, real attack scenarios such as earthquakes and typhoons are instead localised attacks which are investigated only recently. Unlike previous studies, we examine in this paper the resilience of infrastructure networks by focusing on the recovery process from localised attacks. We introduce various preferential repair strategies and found that they facilitate and improve network recovery compared to that of random repairs, especially when population size is uneven at different locations. Moreover, our strategic repair methods show similar effectiveness as the greedy repair. The validations are conducted on simulated networks, and on real networks with real disasters. Our method is meaningful in practice as it can largely enhance network resilience and contribute to network risk reduction.

Empirical Studies on the Network of Social Groups: The Case of Tencent QQ.

BACKGROUND: Participation in social groups are important but the collective behaviors of human as a group are difficult to analyze due to the difficulties to quantify ordinary social relation, group membership, and to collect a comprehensive dataset. Such difficulties can be circumvented by analyzing online social networks. METHODOLOGY/PRINCIPAL FINDINGS: In this paper, we analyze a comprehensive dataset released from Tencent QQ, an instant messenger with the highest market share in China. Specifically, we analyze three derivative networks involving groups and their members-the hypergraph of groups, the network of groups and the user network-to reveal social interactions at microscopic and mesoscopic level. CONCLUSIONS/SIGNIFICANCE: Our results uncover interesting behaviors on the growth of user groups, the interactions between groups, and their relationship with member age and gender. These findings lead to insights which are difficult to obtain in social networks based on personal contacts.

Coverage versus supply cost in facility location: physics of frustrated spin systems.

A comprehensive coverage is crucial for communication, supply, and transportation networks, yet it is limited by the requirement of extensive infrastructure and heavy energy consumption. Here, we draw an analogy between spins in antiferromagnet and outlets in supply networks, and apply techniques from the studies of disordered systems to elucidate the effects of balancing the coverage and supply costs on the network behavior. A readily applicable, coverage optimization algorithm is derived. Simulation results show that magnetized and antiferromagnetic domains emerge and coexist to balance the need for coverage and energy saving. The scaling of parameters with system size agrees with the continuum approximation in two dimensions and the tree approximation in random graphs. Due to frustration caused by the competition between coverage and supply cost, a transition between easy and hard computation regimes is observed. We further suggest a local expansion approach to greatly simplify the message updates which shed light on simplifications in other problems.

Self-sustained clusters and ergodicity breaking in spin models.

Self-sustained spin clusters are analytically linked to ergodicity breaking in fully connected Ising and Sherrington-Kirkpatick (SK) models, relating the less understood spin space to the well understood state space. This correspondence is established through the absence of clusters in the paramagnetic phase, the presence of one dominant cluster in the Ising ferromagnet, and the formation of nontrivial clusters in SK spin glass. Yet unobserved phenomena are also revealed such as a first order phase transition in cluster sizes in the SK ferromagnet. The method could be adapted to investigate other spin models.

From the physics of interacting polymers to optimizing routes on the London Underground.

Optimizing paths on networks is crucial for many applications, ranging from subway traffic to Internet communication. Because global path optimization that takes account of all path choices simultaneously is computationally hard, most existing routing algorithms optimize paths individually, thus providing suboptimal solutions. We use the physics of interacting polymers and disordered systems to analyze macroscopic properties of generic path optimization problems and derive a simple, principled, generic, and distributed routing algorithm capable of considering all individual path choices simultaneously. We demonstrate the efficacy of the algorithm by applying it to: (i) random graphs resembling Internet overlay networks, (ii) travel on the London Underground network based on Oyster card data, and (iii) the global airport network. Analytically derived macroscopic properties give rise to insightful new routing phenomena, including phase transitions and scaling laws, that facilitate better understanding of the appropriate operational regimes and their limitations, which are difficult to obtain otherwise.

Competition for shortest paths on sparse graphs.

Optimal paths connecting randomly selected network nodes and fixed routers are studied analytically in the presence of a nonlinear overlap cost that penalizes congestion. Routing becomes more difficult as the number of selected nodes increases and exhibits ergodicity breaking in the case of multiple routers. The ground state of such systems reveals nonmonotonic complex behaviors in average path length and algorithmic convergence, depending on the network topology, and densities of communicating nodes and routers. A distributed linearly scalable routing algorithm is also devised.

Self-organization in social tagging systems.

Individuals often imitate each other to fall into the typical group, leading to a self-organized state of typical behaviors in a community. In this paper, we model self-organization in social tagging systems and illustrate the underlying interaction and dynamics. Specifically, we introduce a model in which individuals adjust their own tagging tendency to imitate the average tagging tendency. We found that when users are of low confidence, they tend to imitate others and lead to a self-organized state with active tagging. On the other hand, when users are of high confidence and are stubborn to change, tagging becomes inactive. We observe a phase transition at a critical level of user confidence when the system changes from one regime to the other. The distributions of post length obtained from the model are compared to real data, which show good agreement.

Leaders in social networks, the Delicious case.

Finding pertinent information is not limited to search engines. Online communities can amplify the influence of a small number of power users for the benefit of all other users. Users' information foraging in depth and breadth can be greatly enhanced by choosing suitable leaders. For instance in delicious.com, users subscribe to leaders' collection which lead to a deeper and wider reach not achievable with search engines. To consolidate such collective search, it is essential to utilize the leadership topology and identify influential users. Google's PageRank, as a successful search algorithm in the World Wide Web, turns out to be less effective in networks of people. We thus devise an adaptive and parameter-free algorithm, the LeaderRank, to quantify user influence. We show that LeaderRank outperforms PageRank in terms of ranking effectiveness, as well as robustness against manipulations and noisy data. These results suggest that leaders who are aware of their clout may reinforce the development of social networks, and thus the power of collective search.